Existing image search engines index images based on textual information associated with the images, such as surrounding text, anchor text, uniform resource locators (URLs), etc. A user who wants to perform an image search may submit one or more keywords to an image search engine, which may compare the keyword(s) with the textual information of the images and return a search result including a certain number of images to the user. An accuracy of the search result depends on an accuracy of the textual information representing the images and an availability of the textual information of the images. If the textual information is inaccurate or insufficient, the search engine may return irrelevant images in the search result. Furthermore, the search result may include incorrect rankings of the images, with some irrelevant images being placed in high rank positions while certain relevant images being placed in low rank positions.